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ABSTRACT 

The Agency for International Development (AID) 
participant training program was examined to determine its impact on 
developing countries. One hundred and thirty-four senior officials in 
Ghana and Thailand provided 4*^2 reports on: (1) indicators of 
participant achievements; (2) impact- producing characteristics; and 
(3) contributions attributed to participant training. An evaluation 
kit^ or methodology, was field tested in Korea, the Philippines^ and 
Brazil. Three hundred and sixty-four local officials reported 2,871 
critical inciderts. The data clearly showed that AID participants 
trained in the li.S. made significant contributions to the development 
of their countries, and that 58% of these contributions could be 
linked to some specifiable aspect of the participant training. The 
author predicted that participants would continue to make beneficial 
contributions in areas such as agriculture, economic planning, 
health, and public administration. Follow-up studies that would 
provide useful policy guidance were suggested. (Author/GDC) 
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The AID participant training program has provided study 
grants to almost 200,000 professionals, administrators, and 
technical personnel from the developing countries. The vast 
majority of these grants has been for training experiences in 
the United States, ranging from relatively brief observation 
tours through focused work-study programs, to degree courses 
at the graduate and post-graduate levels. The fields in which 
training has been provided have spanned the full spectrum of 
economic and social development needs; the costs have exceeded 
400 million dollars. 

The overwhelming consensus, both here and abroad, is that 
no other form of technical assistance has been more productive 
than this training component. Yet, the lack of techniques for 
systematically measuring outcomes has hampered the program. 
Host country and USAID officials and participant nominees fre- 
quently espouse different views on the "right" training 
strategy to adopt in a given situation. The trade-offs between 
practical experience and paper credentials, between the custom- 
ized programs that can be offered when participants are clustered 
at one training institution and the diversity of perspectives 
they will obtain if dispersed over many institutions are among 
the countless tactical issues that have long been debated but 
not yet resolved. A facility for comparing Approach X with 
Approach Y and reaching a rational conclusion remains a signifi- 
cant need. The participant training activity, like many result- 
oriented endeavors, must have, a well-functioning feedback loop to 
be fully effective. 

Earlier attempts to establish adequate feedback mechanisms 
applied criteria of program effectiveness that were intrinsically 
fragmentary and soft. Indices based on the participants' ex- 
pressed satisfactions with the training they received fell short 
of the need. If the purpose of participant training is to ac- 
celerate development in the recipient countries, an index of 
tangible contributions to development would serve as an adequate 
measure of program success. The need was for feedback on training- 
linked development gains. 
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This project was divided into two phases. Phase I focused 
on the development of suitable impact assessment procedures; 
Phase II emphasized testing the methodology in three independent 
field assessments, then incorporating the revised procedures. 

Two basic requirements must be met in an impact assessment. 
The contributions that have been made to national goals must be 
identified ; in addition, these contributions must be attributed 
to a special input, such as participant training. The AIR ap- 
proach was based on two strategic decisions. The first was to 
concentrate the search for suitable indicators within a fixed 
segment of the long chain of events that links the training in- 
puts to the ultimate goal of national development gains. The 
following schematic highlights the segitient on which the search 
was focused. 
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The second decision was to look foi indicators in this seg- 
ment with a search process precisely opposite to that used in 
earlier participant follow-up studies. Rather than beginning 
with the outcomes of the training program and looking for their 
effects in or on the institution, we began with the identification 
of visible improvements or achievements, and then traced these 
"backward" to their antecedents. The initial step was to identify 
events that might prove serviceable as indicators for assessment. 
A three-step process was used to develop the required indicators. 
The first step was to obtain from a samplre of former participants 
and their supervisors reports (critical incidents) of specific im- 
provements that have occurred in the output of the institution or 
in its operations since the participant ' s return. The second step 
was to seek from the same respondents such evidence as they may be 



able to cite concerning the relationships, if any, of these 
•achievements to experiences during participant training* The 
final step was to deduce from these data the types of achieve- 
ments that most effectively straddle the contribution- 
attribution dilemma, and to fashion these into prototype indi- 
cators for impact assessment. The product was a master list of 
indicators classified three ways: (1) indicators of participant 
achievements; (2) impact-producing characteristics; (3) attribu- 
tions to participant training. 

The feasibility study was conducted in Ghana and Thailand. 
Senior officials were interviewed in fields such agriculture , 
health, revenue, and transportation. One hundred thirty-four 
(134) people were interviewed; 4 92 reports were collected. The 
reports were sorted into the three categories mentioned above; 
the sorting resulted in 20 separate groupings of participant 
achievements, 14 impact-producing characteristics end 5 groups 
of attribution information representing decreasing attributa- 
bility to participant training. Phase II was a test of the 
methodology in three developing coimtries and the preparation of 
an "evaluation kit" that eventuaMy could be used in local opera- 
tional assessments . 

The three field assessments were conducted in Korea, the 
Philippines, and Brazil. With each successive study, additional 
responsibility for the data collection was given to local per- 
sonnel. Officials were interviewed in the agricultural and the 
economic planning/public administration sectors. Three hundred 
sixty-four (364) officials were interviewed; 2,871 incidents were 
collected. 

The data clearly show that USAID participants ' have made 
and are making significant contributions to the development of 
their countries. There is clear evidence that a substantial 
portion of these contributions (58%) can be linked to some 
specifiable aspect of the participant experience. And the 



benefits will continue, to accrue. The individual participants 
will make new contributions, and, in addition, they serve as 
multipliers of institutional capability. The people chosen to 
represent their country in the participant training program 
were identified as the most promising candidates; there was 
nothing random about their selection. Our data provide no 
estimate at all of the contributions which would have beeja ex- 
pected in the absence of the participant program, but common 
sense tells us that some would have occurred. 

In Phase I a small number of impact-enabler combinations 
accounted for most of the reported events. This characteristic 
has great practical utility; one can design efficient procedures 
by focusing on specific combinations of high potency. The com- 
parability of the two sectors provided some interesting results. 
Economic planning and agriculture are distinct and reasonably 
well-defined fields, but appear on the surface to differ con- 
siderably. There is little reason to expect similarities in the 
nature of 'impact , in the enabler- impact combination, or in the 
pattern of attribution. " Some similarity, was found. The correla- 
tion between the number of reports in each of the twenty impact 
categories for the two sectors is . 64 (41% of the variance in 
the nature of impact is common to the two) ; the correlation was 
.74 for the enabling characteristics (55% of the variance is 
common), and a correlation of .57 in the frequency of attribution 
(32% of the variance is common). The commonalitv of the two sec- 
tors is considerable, approxijnating 50% of the variance. 

There are three major types of follow-up studies that can 
supply useful policy guidance. Type I would be open-ended, 
diagnostic assessments that would be carried out periodically 
for specified training activities. Each study would focus on a 
single type of training activity and establish the nature of im- 
pacts that graduates of this program typically are (or are not) 
achieving. Type II studies would be comparative evaluations of 
alternative patterns of implementation that have been tried at 
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varying times, by separate trc:ining centers. They would be 
designed to test specific hypotheses about specified program 
characteristics. Type III studies would aim for policy guid- 
ance on basic programming characteristics that affect most or 
all of the separate training activities that comprise the pro- 
gram. It is anticipated that Type III studies would not be 
based on new data collection, but rather conducted by appro- 
priate analyses of the aggregate data base that has been 
provided by the more narrowly focused assessments already 
completed. 



Attachments 



The attechments included in the following pages illustrate some 
of the major features described in the presentation. 

1 . Participant achievements . These represent the nature of the 
impact that was the participant's end product or action. 

2. I mpact-produc i ng ch^r?oter i st i cs , Theje represent the various 
paths the participants took to bring about the 20 achievements; 
the reports are reexamined from this point of view. These 
characteristics are defined as the specific skill, attitude or 
other resource that the participant brought to the situation 
to effect the impact reported. 

3. Attributions to traininc • These represent decreasing attr i butab i I i ty 
to participent training; i liey permit decisions on which of the 
sexquences are initiated by an experience provided by participant 
training. 

4. Illustrative critical incidents . Thb,.« eight reports demonstrate 
the classification procedure used for the more than 3,300 inci- 
dents collected during Phase I and Phase II. 

b. Table I. Percentage of Reports Attributed to Participant Training : 
High Freguency Con'bi nations in Agriculture . 



6. Table II. Perccn-tage of Reports Attributed to Participant Training : 
High Frequency Corr^bi nations in Economic Planning/Public Administration . 



! a I o n C'f Types of Participant Achievements 

IMPACT ON DEVELOPMENT TARGETS 

1. InfU^onced development strategies or emphases, or a specific invesT- 
:neiit drclslor^. 

2. Introduced a new agricultural, industrial, or commercial enterprise 
the country. 

3. Developed a loca I capabi I ity for an activity formerly dependent on 
externa I resources . 

4. Discovered a solution or a more promising approach to a significant 
development problem. 

5. Stimulated the more wide^spread adoption of a preferred practice or 
other, desired public response. 

IMPACT ON INSTITUTIONAL OUTPUTS 

6. Initiated a new service or program , 

7. Raised standards of products or services provided. 

8. Changed rules or procedures to be more responsive to needs of clients . 

9. Avoided disruption of service by timely action , despite difficulties or risk. 

10. Performed task that required specia I effort or ski I I . 

11. Improved or expanded dissemination programs, techniques. 

IMPACT ON OUTSIDE SUPPORTS 

12. Expanded institution's authority, status, or charter . 

13. Developed more effective working relationships with local agencies or 
sources of external aid. 

IMPACT ON INTERNAL OPERATIONS 

14. Introduced or expanded the use of analytic, data-based management aids . 

15. Introduced cost- or time-saving measures , ideas. 

16. Imposed tighter structure or controls on staff or vendor performance. 

17. Improved the allocation or organization of responsibilities and functions. 

18. Upgraded the caliber, capabilities, or morale of the staff. 

19. Upgraded physical facilities or equipment . 
gpy(^" 20. Improved record-keeping or ^Information retrieval systems. ^ 



Impact-Producing Characteristics 



PROGRAM 

A. Technical capabilities, sophistication. 

B. Awareness of other possibilities, approaches. 

C. Appreciation of nature and magnitude of inputs required. 

DOCTRINE 

D. Acceptance of new or expanded objectives. 

E. Commitment to principles, convictions, 

LEADERSHIP 

F. Willingness to take responsibility, act. 

G. Data orientation 

H. Goal orientation 

RESOURCE MANAGEMENT 

I. Efficiency orientat icn- 
J. Skill in human relations 

K. Familiarity with equipment. 

L. Familiarity with workable operating routines. 
LINKAGES 

M. Access to external sources of Information or help* 
N. Credibility and credentials. 
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Attributions to Training 



!• Reasonably clear - :ut links to training 



a. Specific technique or theory r^pplied 

b. Specific practice or model adopted 

c. U.S. source or product applied 

d. Practical jcb experience cited 

e. I nci dental ski I I learned 

f . Credentials appi ied 

g. Before-after changes observed 



lU Probable links to training 

a. Technical background cited 

b. U.S. work style cited 

c. Timing of the event 

d. Requirement for technical knowledge 

e. Conformity of approach to U.S. standard 

II. Possi b le links to training 

a. Claim of increaseo self-assurance 

b. Claim of attitude change 

IV. Doubtf u I links 1o training 



a. 



b. 



Persona I character i sti cs 
Clever ideas 



V. 



No I i nks 



to training 
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ILLUSTRATIVE CRITICAL INCIDENTS 



The following illustrative reports were collected by local interview 
gathered critical incidents for an impact assessment of participant 



Illustration #1 
Report 

Recommended and drafted the new barangay zone system for the 
which reduced th^ zone area to a maximum of 50 families. This meL 
easier for the barangay to meet a quorum. The old system was by i 
of over 200 families which was found to be ineffective. 

What was the outcome ? 

It is now easy for the barangay to meet to transact business. T 
closer fellowship and the spirit of brotherhood was evident. 

How was it achieved ? 

He was aware of additional and better ways of achieving his go^ 
Attribution evidence ? 

AID trainee: 1970 in Taiwan: "Planning and Management of Rujt 
Development. " 

Impact 

Classification: Category 1 Characteristic H Attribution Ild 



Illustration #2 
Report 

Iiitroduced concept of the amnesty in 1974; no suits brought aga 
guiif:y tax evaders prior to this time. It was necessary because of 
amount of unreported income and the government needed revenue, 
now file all assets and liabilities; this creates an open network fo. 
Ing true income. Of approximately 600,000 taxable incomes in thi 
400, 000 took advantage of amnesty rather than risk investigation i 
records . 
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What was the outcome? 



Those who took advantage of amnesty pay tax no higher than 20%, 
How was It achieved ? 

As Commissioner, the participant directed development of this plan; it 
became a Presidential decree 

Attribution evidence ? 

Experience; personal resourcefulness. 

Impact 

Classification: Category 1 Characteristic __B Attribution V 



Illustration #3 
Report 

Convinced town mayors of two provinces to set up livestock markets that 
have proper weighing scales, fences^ facilities for storage, standard prices, 
etc. There were only six livestock markets In these two provinces. Because 
of this lack of livestock markets, there were no standard prices in buying and 
selling of livestock. 

What was the outcome ? 

In a span of one year, seven additional livestock marketfi were set up in 
these two provinces. This resulted in fair trading (standard prices were - 
set), and thus led to increased income for the town. 

How was it achieved ? 

Contacted town mayors to directly convince them to set up these livestock 
markets. Persuaded people to help him in this Job. Personal initiative and 
his awareness of his duties as chief of the marketing unit motivated him, too, 
to undertake this project. 

Attribution evidence ? 

Observation of livestock markets In Texas, experiences In his job as 
chief of the marketing unit. 

Impact 

Classification: Category 1 Characteristic C Attribution Ih 



Illustration #4 



Report 

Organized store cooperatives for farm supplies and household needs in 
the barrios. Cooperative stores in the towns most often did not have in 
their stocks what the farmers needed. The barrio cooperatives now sell 
what the farmers need on the farms. 

What was the outcome ? 

It is now easier for the farmers to purchase their farm and household 
supplies right in their own places. They save a lot in terms of time and 
money. 

How was it achieved ? 

Familiarity with the operating coops in Taiwan which he believed would 
work in own country. 

Attribution evidence ? 

Observation and study tours on rural development as AID trainee in 
Taiwan, 1966. 

Impact 

Classification: Category _4 Characteristic M Attribution lb 



Illu stration \\S 
Report 

Introduced removal of registration plates of cars parked In no parking 
areas. In the past parked cars blocked two lanes causing traffic jams. 
This was necessary to ease traffic and to discourage parking at no parking 
areas. 

What was the outcome ? 

Parked cars occupy only one lane; helped case traffic a little. 
How was it achieved ? 
Concern for efficiency. 
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Attribution evidence ? 

His training on revenue administration, U.S. 1959-60. 
Impact 

Classification: Category 5 Characteristic M Attribution lb 

Illustration #6 
:? >port 

Developed training courses on agricultural cooperatives for use in SE 
Asian countries. His agency is the innovator in the area of cooperatives 
and provides assistance in this area to other Asian countries. 

What was the outcome ? 

The courses were accepted by the international cooperative organization 
and project has been submitted for funding. 

How was It achieved ? 

Commitment to achieving agency goals for cooperatives. 
Attribution evidence ? 

Observed needs on many trips to SE Asia on business. 

Impact 

Classification: Category 6 Characteristic H Attribution V 

Illustration #7 
Report 

Introduced the use of actual designs for irrigation projects which cover 
computations and other work necessary to arrive at grades, sizes, and lo- 
cation of irrigation projects. This was necessary because all the details 
of the project could be viewed on the face of the papers. 

What was the outcome? 



Came out with the final map which contains the plans, profiles, and 
specifications which guided construction workers. 



How was it achieved? 



After coming from Colorado on the inservice training with the U.S. 
Bureau of Reclamation, informed Division Chief of actual on-the-job 
training and on observations on ongoing projects in the4J.S. Division 
Chief expected an output from knowledge obtained and assigned him the 
Agno River Project in 1950. 

Attribution evidence ? 

InserviCi. training from Colorado, U.S. Bureau of Reclamation, and by 
the U.S. Reclamation manual. 

Impact 

Classification: Category. 6 Characteristic M Attribution la 



Illustration #8 
Report 

Introduced the improvised incubator for seed germination in the nursery. 
The old incubators used were single flat form. The improvised version is 
composed of several layers but provided with electric lights for each layer 
for faster germination or response of the seed. This was necessary to 
cope with the demands of the consumers. 

What was the outcome ? 

An incubatoi with around 10-15 layers was being used to germinate seeds 
to be distributed for the green revolutional program and is still being used 
at the nursery. 

How was it achieved ? 

With strong determination to be able to cope with the demands of con- 
sumers and having in mind the incubators used at Mississipl State University. 
Devised a way of copying the design because the machine is very expensive. 

Attribution evidence ? 

Masters in B.S. Agriculture at the Mississippi State University. 
Impact 

Classification: Category 7 Characteristic M Attribution Ic 
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Table 1 

Percentage of Reports Attributed to Participant Training: 
High Frequency Combinations in Agriculture 



c 

w o 

D^-H cjin oao a orH+J 

o rH g r-^ -H w •r^ .r^ c o c w 

a:} v^ c o « o o EC 

• O -H O C Oi 'iH 

aooaorirJO^Hdtj'o 







< 




u 


Q 




o 


a: H 








2; 




1. 


Development Decisions 
























2. 


New Enterprises 


























3. 


Local Capabilities 


























4. 


Discoveries/Solutions ^7 


60 




















33 58 


5. 


Public Adoption 


100 


100 




















18 100 


o • 


New Programs 


71 


oo 
oo 
















100 




31 84 


7. 


Higher Standards 


88 






















16 88 


8. 


Client Needs 


























9, 


Timely Actions 


























10. 


Denanding Tasks 


59 


80 








70 








76 




103 68 


11. 


More Dissemination 


49 


63 


40 


13 












45 


70 


133 51 


12. 


Institutional Charter 
























13. 


Outside Relations 


57 














33 




38 




44 43 


14. 


Data-Based Aids 


55 


50 








52 












53 53 


15. 


Cost Savings 














67 










9 67 


16. 


Tighter Controls 


























17. 


Organiz. Structure 


























18. 


Better Staff 


64 


38 


50 




75 






22 




67 


38 


113 48 


19. 


Equipment 






50 












73 






34 68 


20. 


Record-Keeping 














33 






69 




22 55 




No. of Reports 


231 


80 


32 


8 


12 


46 


18 


36 


26 


84 


36 


609 




% Attrib. 


60 


68 


47 


13 


75 


61 


50 


25 


73 


63 


58 


59 



0) 
i-C 

rd 
•H 



4J 




0) 




c 




4J 




a 


o 


u 


•H 






o 


dP U 


o 




0^ 


4J 


u 


o' 


a 




u 


2 


« 


< 




Table II 

Percentage of Reports Attributed to Participant Training: 
High Frequency Combinations in EP/PA 
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